Since debhelper/10.3, there has been a number of performance related changes. The vast majority primarily improves bulk performance or only have visible effects at larger input sizes. Most visible cases are:

• dh + dh_* now scales a lot better for large number of binary packages. Even more so with parallel builds.
• Most dh_* tools are now a lot faster when creating many directories or installing files.
• dh_prep and dh_clean now bulk removals.
• dh_install can now bulk some installations. For a concrete corner-case, libssl-doc went from approximately 11 seconds to less than a second. This optimization is implicitly disabled with exclude (among other).
• dh_installman now scales a lot better with many manpages. Even more so with parallel builds.
• dh_installman has restored its performance under fakeroot (regression since 10.2.2)

For debhelper, this mostly involved:

• avoiding fork+exec of commands for things doable natively in perl. Especially, when each fork+exec only process one file or dir.
• bulking as many files/dirs into the call as possible, where fork+exec is still used.
• caching / memorizing slow calls (e.g. in parts of pkgfile inside Dh_Lib)
• adding an internal API for dh to do bulk check for pkgfiles. This is useful for dh when checking if it should optimize out a helper.
• and, of course, doing things in parallel where trivially possible.

How to take advantage of these improvements in tools that use Dh_Lib:

• If you use install_ file,prog,lib,dir , then it will come out of the box. These functions are available in Debian/stable. On a related note, if you use doit to call install (or mkdir ), then please consider migrating to these functions instead.
• If you need to reset owner+mode (chown 0:0 FILE + chmod MODE FILE), consider using reset_perm_and_owner. This is also available in Debian/stable.
  • CAVEAT: It is not recursive and YMMV if you do not need the chown call (due to fakeroot).
• If you have a lot of items to be processed by a external tool, consider using xargs(). Since 10.5.1, it is now possible to insert the items anywhere in the command rather than just in the end.
• If you need to remove files, consider using the new rm_files function. It removes files and silently ignores if a file does not exist. It is also available since 10.5.1.
• If you need to create symlinks, please consider using make_symlink (available in Debian/stable) or make_symlink_raw_target (since 10.5.1). The former creates policy compliant symlinks (e.g. fixup absolute symlinks that should have been relative). The latter is closer to a ln -s call.
• If you need to rename a file, please consider using rename_path (since 10.5). It behaves mostly like mv -f but requires dest to be a (non-existing) file.
• Have a look at whether on_pkgs_in_parallel() / on_items_in_parallel() would be suitable for enabling parallelization in your tool.
  • The emphasis for these functions is on making parallelization easy to add with minimal code changes. It pre-distributes the items which can lead to unbalanced workloads, where some processes are idle while a few keeps working.

Credits: I would like to thank the following for reporting performance issues, regressions or/and providing patches. The list is in no particular order:

• Helmut Grohne
• Kurt Roeckx
• Gianfranco Costamagna
• Iain Lane
• Sven Joachim
• Adrian Bunk
• Michael Stapelberg

Should I have missed your contribution, please do not hesitate to let me know.
Filed under: Debhelper, Debian

Since debhelper/10.3, there has been a number of performance related changes. The vast majority primarily improves bulk performance or only have visible effects at larger input sizes. Most visible cases are:

• dh + dh_* now scales a lot better for large number of binary packages. Even more so with parallel builds.
• Most dh_* tools are now a lot faster when creating many directories or installing files.
• dh_prep and dh_clean now bulk removals.
• dh_install can now bulk some installations. For a concrete corner-case, libssl-doc went from approximately 11 seconds to less than a second. This optimization is implicitly disabled with exclude (among other).
• dh_installman now scales a lot better with many manpages. Even more so with parallel builds.
• dh_installman has restored its performance under fakeroot (regression since 10.2.2)

For debhelper, this mostly involved:

• avoiding fork+exec of commands for things doable natively in perl. Especially, when each fork+exec only process one file or dir.
• bulking as many files/dirs into the call as possible, where fork+exec is still used.
• caching / memorizing slow calls (e.g. in parts of pkgfile inside Dh_Lib)
• adding an internal API for dh to do bulk check for pkgfiles. This is useful for dh when checking if it should optimize out a helper.
• and, of course, doing things in parallel where trivially possible.

How to take advantage of these improvements in tools that use Dh_Lib:

• If you use install_ file,prog,lib,dir , then it will come out of the box. These functions are available in Debian/stable. On a related note, if you use doit to call install (or mkdir ), then please consider migrating to these functions instead.
• If you need to reset owner+mode (chown 0:0 FILE + chmod MODE FILE), consider using reset_perm_and_owner. This is also available in Debian/stable.
  • CAVEAT: It is not recursive and YMMV if you do not need the chown call (due to fakeroot).
• If you have a lot of items to be processed by a external tool, consider using xargs(). Since 10.5.1, it is now possible to insert the items anywhere in the command rather than just in the end.
• If you need to remove files, consider using the new rm_files function. It removes files and silently ignores if a file does not exist. It is also available since 10.5.1.
• If you need to create symlinks, please consider using make_symlink (available in Debian/stable) or make_symlink_raw_target (since 10.5.1). The former creates policy compliant symlinks (e.g. fixup absolute symlinks that should have been relative). The latter is closer to a ln -s call.
• If you need to rename a file, please consider using rename_path (since 10.5). It behaves mostly like mv -f but requires dest to be a (non-existing) file.
• Have a look at whether on_pkgs_in_parallel() / on_items_in_parallel() would be suitable for enabling parallelization in your tool.
  • The emphasis for these functions is on making parallelization easy to add with minimal code changes. It pre-distributes the items which can lead to unbalanced workloads, where some processes are idle while a few keeps working.

Credits: I would like to thank the following for reporting performance issues, regressions or/and providing patches. The list is in no particular order:

• Helmut Grohne
• Kurt Roeckx
• Gianfranco Costamagna
• Iain Lane
• Sven Joachim
• Adrian Bunk
• Michael Stapelberg

Should I have missed your contribution, please do not hesitate to let me know.
Filed under: Debhelper, Debian

In case you're for example using Alpine Linux 3.6 based docker images, and you've been passing through environment variable names with dots, you might miss them now in your actual environment. It seems that with busybox 1.26.0 the busybox ash got a lot stricter regarding validation of environment variable names and now you can no longer pass through variable names with dots in them. They just won't be there. If you've been running ash interactively you could not add them in the past, but until now you could do something like this in your Dockerfile

ENV foo.bar=baz

and later on accces a variable "foo.bar". bash still allows those invalid variable names and is way more tolerant. So to be nice to your devs, and still bump your docker image version, you can add bash and ensure you're starting your application with /bin/bash instead of /bin/sh inside of your container. Links

• A bugreport at Alpine Linux.
• Relevant part of the POSIX standard.
• Spring documentation regarding relaxed binding of environment variables - you're not forced to use dots

We bought a bunch of very cheap low end HPE DL120 server. Enough to warrant a completely automated installation setup. Shouldn't be that much of a deal, right? Get dnsmasq up and running, feed it a preseed.cfg and be done with it. In practise it took us more hours then we expected. Setting up the hardware Our hosts are equipped with an additional 10G dual port NIC and we'd like to use this NIC for PXE booting. That's possible, but it requires you to switch to UEFI boot. Actually it enables you to boot from any available NIC. Setting up dnsmasq We decided to just use the packaged debian-installer from jessie and do some ugly things like overwritting files in /usr/lib via ansible later on. So first of all install debian-installer-8-netboot-amd64 and dnsmasq, then enroll our additional config for dnsmasq, ours looks like this:

domain=int.foobar.example
dhcp-range=192.168.0.240,192.168.0.242,255.255.255.0,1h
dhcp-boot=bootnetx64.efi
pxe-service=X86-64_EFI, "Boot UEFI PXE-64", bootnetx64.efi
enable-tftp
tftp-root=/usr/lib/debian-installer/images/8/amd64/text
dhcp-option=3,192.168.0.1
dhcp-host=00:c0:ff:ee:00:01,192.168.0.123,foobar-01

Now you've to link /usr/lib/debian-installer/images/8/amd64/text/bootnetx64.efi to /usr/lib/debian-installer/images/8/amd64/text/debian-installer/amd64/bootnetx64.efi. That got us of the ground and we had a working UEFI PXE boot that got us into debian-installer. Feeding d-i the preseed file Next we added some grub.cfg settings and parameterized some basic stuff to be handed over to d-i via the kernel command line. You'll find the correct grub.cfg in /usr/lib/debian-installer/images/8/amd64/text/debian-installer/amd64/grub/grub.cfg. We added the following two lines to automate the start of the installer:

set default="0"
set timeout=5

and our kernel command line looks like this:

 linux    /debian-installer/amd64/linux vga=788 --- auto=true interface=eth1 netcfg/dhcp_timeout=60 netcfg/choose_interface=eth1 priority=critical preseed/url=tftp://192.168.0.2/preseed.cfg quiet

Important points:

• tftp host IP is our dnsmasq host.
• Within d-i we see our NIC we booted from as eth1. eth0 is the shared on-board ilo interface. That differs e.g. within grml where it's eth2.

preseeed.cfg, GPT and ESP One of the most painful points was the fight to find out the correct preseed values to install with GPT to create a ESP (EFI system partition) and use LVM for /. Relevant settings are:

# auto method must be lvm
d-i partman-auto/method string lvm
d-i partman-lvm/device_remove_lvm boolean true
d-i partman-md/device_remove_md boolean true
d-i partman-lvm/confirm boolean true
d-i partman-lvm/confirm_nooverwrite boolean true
d-i partman-basicfilesystems/no_swap boolean false
# Keep that one set to true so we end up with a UEFI enabled
# system. If set to false, /var/lib/partman/uefi_ignore will be touched
d-i partman-efi/non_efi_system boolean true
# enforce usage of GPT - a must have to use EFI!
d-i partman-basicfilesystems/choose_label string gpt
d-i partman-basicfilesystems/default_label string gpt
d-i partman-partitioning/choose_label string gpt
d-i partman-partitioning/default_label string gpt
d-i partman/choose_label string gpt
d-i partman/default_label string gpt
d-i partman-auto/choose_recipe select boot-root-all
d-i partman-auto/expert_recipe string \
boot-root-all :: \
538 538 1075 free \
$iflabel  gpt   \
$reusemethod    \
method  efi   \
format    \
. \
128 512 256 ext2 \
$defaultignore    \
method  format   format    \
use_filesystem    filesystem  ext2   \
mountpoint  /boot   \
. \
1024 4096 15360 ext4 \
$lvmok    \
method  format   format    \
use_filesystem    filesystem  ext4   \
mountpoint  /   \
. \
1024 4096 15360 ext4 \
$lvmok    \
method  format   format    \
use_filesystem    filesystem  ext4   \
mountpoint  /var   \
. \
1024 1024 -1 ext4 \
$lvmok    \
method  format   format    \
use_filesystem    filesystem  ext4   \
mountpoint  /var/lib   \
.
# This makes partman automatically partition without confirmation, provided
# that you told it what to do using one of the methods above.
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman-md/confirm boolean true
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true
# This is fairly safe to set, it makes grub install automatically to the MBR
# if no other operating system is detected on the machine.
d-i grub-installer/only_debian boolean true
d-i grub-installer/with_other_os boolean true
d-i grub-installer/bootdev  string /dev/sda

I hope that helps to ease the processes to setup automated UEFI PXE installations for some other people out there still dealing with bare metal systems. Some settings took us some time to figure out, for example d-i partman-efi/non_efi_system boolean true required some searching on codesearch.d.n (amazing ressource if you're writing preseed files and need to find the correct templates) and reading scripts on git.d.o where you'll find the source for partman-* and grub-installer. Kudos Thanks especially to P.P. and M.K. to figure out all those details.

People using Chrome might have already noticed that some internal certificates created without a SubjectAlternativeName extension fail to verify. Finally the Google Chrome team stepped forward, and after only 17 years of having SubjectAlternativeName as the place for FQDNs to verify as valid for a certificate, they started to ignore the commonName. See also https://www.chromestatus.com/feature/4981025180483584. Currently Debian/stretch still has Chromium 57 but Chromium 58 is already in unstable. So some more people might notice this change soon. I hope that everyone who maintains some broken internal scripting to maintain internal CAs now re-reads the OpenSSL Cookbook to finally fix this stuff. In general I recommend to base your internal CA scripting on easy-rsa to avoid making every mistake in certificate management on your own.

Must be the irony of life that I was about to give up the TclCurl Debian package some time ago, and now I'm using it again for some very old and horrible web scraping code. The world moved on to https but the Tcl http package only supports unencrypted http. You can combine it with the tls package as explained in the Wiki, but that seems to be overly complicated compared to just loading the TclCurl binding and moving on with something like this:

package require TclCurl
# download to a variable
curl::transfer -url https://sven.stormbind.net -bodyvar page
# or store it in a file
curl::transfer -url https://sven.stormbind.net -file page.html

Now the remaining problem is that the code is unmaintained upstream and there is one codebase on bitbucket and one on github. While I fed patches to the bitbucket repo and thus based the Debian package on that repo, the github repo diverted in a different direction.

After a few weeks of running Exodus on my moto g falcon, I've now done again the full wipe and moved on to LineageOS nightly from 20170213. Though that build is no longer online at the moment. It's running smooth so far for myself but there was an issue with the Google Play edition of the phone according to Reddit. Since I don't use gapps anyway I don't care. The only issue I see so far is that I can not reach the flash menu in the camera app. It's hidden behind a grey bar. Not nice but not a show stopper for me either.

Deng Xiaoping (who succeeded Mao) expounded this view and obviously did not depend on a minority to succeed. In open source software projects we often find ourselves implementing features of interest to a minority of users to keep our software relevant to a larger audience.

As previously mentioned I contribute to the NetSurf project and the browser natively supports numerous toolkits for numerous platforms. This produces many challenges in development to obtain the benefits of a more diverse user base. As part of the recent NetSurf developer weekend we took the opportunity to review all the frontends to make a decision on their future sustainability.

Each of the nine frontend toolkits were reviewed in turn and the results of that discussion published. This task was greatly eased because we we able to hold the discussion face to face, over time I have come to the conclusion some tasks in open source projects greatly benefit from this form of interaction.

Coding and day to day discussions around it can be easily accommodated va IRC and email. Decisions affecting a large area of code are much easier with the subtleties of direct interpersonal communication. An example of this is our decision to abandon the cocoa frontend (toolkit used on Mac OS X) against that to keep the windows frontend.

The cocoa frontend was implemented by Sven Weidauer in 2011, unfortunately Sven did not continue contributing to this frontend afterwards and it has become the responsibility of the core team to maintain. Because NetSuf has a comprehensive CI system that compiles the master branch on every commit any changes that negatively affected the cocoa frontend were immediately obvious.

Thus issues with the compilation were fixed promptly but because these fixes were only ever compile tested and at some point the Mac OS X build environments changed resulting in an application that crashes when used. Despite repeatedly asking for assistance to fix the cocoa frontend over the last eighteen months no one had come forward.

And when the topic was discussed amongst the developers it quickly became apparent that no one had any objections to removing the cocoa support. In contrast the windows frontend, which despite having many similar issues to cocoa, we decided to keep. These were almost immediate consensus on the decision, despite each individual prior to the discussion not advocating any position.

This was a single example but it highlights the benefits of a disparate development team having a physical meeting from time to time. However this was not the main point I wanted to discuss, this incident highlights that supporting a feature only useful to a minority of users can have a disproportionate cost.

The cost of a feature for an open source project is usually a collection of several factors:

Developer time

Arguably the greatest resource of a project is the time its developers can devote to it. Unless it is a very large, well supported project like the Kernel or libreoffice almost all developer time is voluntary.

Developer focus

Any given developer is likely to work on an area of code that interests them in preference to one that does not. This means if a developer must do work which does not interest them they may loose focus and not work on the project at all.

Developer skillset

A given developer may not have the skillset necessary to work on a feature, this is especially acute when considering minority platforms which often have very, very few skilled developers available.

Developer access

It should be obvious that software that only requires commodity hardware and software to develop is much cheaper than that which requires special hardware and software. To use our earlier example the cocoa frontend required an apple computer running MAC OS X to compile and test, this resource was very limited and the project only had access to two such systems via remote desktop. These systems also had to serve as CI builders and required physical system administration as they could not be virtualized.

Support

Once a project releases useful software it generally gains users outside of the developers. Supporting users consumes developer time and generally causes them to focus on things other than code that interests them.

While most developers have enough pride in what they produce to fix bugs, users must always remember that the main freedom they get from OSS is they recived the code and can change it themselves, there is no requirement for a developer to do anything for them.

Resources

A project requires a website, code repository, wiki, CI systems etc. which must all be paid for. Netsurf for example is fortunate to have Pepperfish look after our website hosting at favorable rates, Mythic beasts provide exceptionally good rates for the CI system virtual machine along with hardware donations (our apple macs were donated by them) and Collabora for providing physical hosting for our virtual machine server.

Despite these incredibly good deals the project still spends around 200gbp (250usd) a year on overheads, these services obviously benefit the whole project including minority platforms but are generally donated by users of the more popular platforms.

The benefits of a feature are similarly varied:

Developer learning

A developer may implement a feature to allow them to learn a new technology or skill

Project diversity

A feature may mean the project gets built in a new environment which reveals issues or opportunities in unconnected code. For example the Debian OS is built on a variety of hardware platforms and sometimes reveals issues in software by compiling it on big endian systems. These issues are often underlying bugs that are causing errors which are simply not observed on a little endian platform.

More users

Gaining users of the software is often a benefit and although most OSS developers are contributing for personal reasons having their work appreciated by others is often a factor. This might be seen as the other side of the support cost.

In the end the maintainers of a project often have to consider all of these factors and more to arrive at a decision about a feature, especially those only useful to a minority of users. Such decisions are rarely taken lightly as they often remove another developers work and the question is often what would I think about my contributions being discarded?

As a postscript, if anyone is willing to pay the costs to maintain the NetSurf cocoa frontend I have not removed the code just yet.

For CentOS 4 to CentOS 6 we used pam_ldap to restrict host access to machines, based on groupOfUniqueNames listed in an openldap. With RHEL/CentOS 6 RedHat already deprecated pam_ldap and highly recommended to use sssd instead, and with RHEL/CentOS 7 they finally removed pam_ldap from the distribution. Since pam_ldap supported groupOfUniqueNames to restrict logins a bigger collection of groupOfUniqueNames were created to restrict access to all kind of groups/projects and so on. But sssd is in general only able to filter based on an "ldap_access_filter" or use the host attribute via "ldap_user_authorized_host". That does not allow the use of "groupOfUniqueNames". So to allow a smoth migration I had to configure sssd in some way to still support groupOfUniqueNames. The configuration I ended up with looks like this:

[domain/hostacl]
autofs_provider = none 
ldap_schema = rfc2307bis
# to work properly we've to keep the search_base at the highest level
ldap_search_base = ou=foo,ou=people,o=myorg
ldap_default_bind_dn = cn=ro,ou=ldapaccounts,ou=foo,ou=people,o=myorg
ldap_default_authtok = foobar
id_provider = ldap
auth_provider = ldap
chpass_provider = none
ldap_uri = ldaps://ldapserver:636
ldap_id_use_start_tls = false
cache_credentials = false
ldap_tls_cacertdir = /etc/pki/tls/certs
ldap_tls_cacert = /etc/pki/tls/certs/ca-bundle.crt
ldap_tls_reqcert = allow
ldap_group_object_class = groupOfUniqueNames
ldap_group_member = uniqueMember
access_provider = simple
simple_allow_groups = fraappmgmtt
[sssd]
domains = hostacl
services = nss, pam
config_file_version = 2

Important side note: With current sssd versions you're more or less forced to use ldaps with a validating CA chain, though hostnames are not required to match the CN/SAN so far. Relevant are:

• set the ldap_schema to rfc2307bis to use a schema that knows about groupOfUniqueNames at all
• set the ldap_group_object_class to groupOfUniqueNames
• set the the ldap_group_member to uniqueMember
• use the access_provider simple

In practise what we do is match the member of the groupOfUniqueNames to the sssd internal group representation. The best explanation about the several possible object classes in LDAP for group representation I've found so far is unfortunately in a german blog post. Another explanation is in the LDAP wiki. In short: within a groupOfUniqueNames you'll find a full DN, while in a posixGroup you usually find login names. Different kind of object class requires a different handling. Next step would be to move auth and nss functionality to sssd as well.

Recently some of my coworkers and I experienced an issue with using the upper left touchpad button on our Dell Latitude E7470 and similar laptops (E5xxx from the current generation). Some time in January we could no longer hold down this button and select text with the touchpad. Using the left button below the touchpad still worked. This hit my coworker running Fedora and myself running Debian/stretch. So I first thought that it's likely a libinput issue (same version in Debian/stretch and Fedora and I recently pulled that in as an update), somehow blacklisting the upper left key because it's connected to the trackpoint. So I filled #99594 upstream. While this was not very helpful at first, and according to Peter very unlikely to be related to libinput, another coworker using Debian/jessie found this issue to hit him when he upgraded the backports kernel in use from 4.8 to 4.9. That finally led to the conclusion that it's a bug in the Linux alps driver, which is already fixed in 4.10 and probably 4.9.6. Until the Debian kernel team pulls in a fresh 4.9 point release I'm using 4.10-rc6 from experimental. For Debian/jessie + backports kernel user it might be more convenient to just stay at 4.8 in case this issue annoys you. Kudos to Peter, Benjamin, TW and WW for the help in locating the origin of this issue! Lessons learned:

• I should've started with the painful downgrade of xorg and libinput via snapshot.d.o before opening the bugreport.
• A lot more of the touchpad related hardware support is nowadays in the kernel and not in the xorg layer. Either that was just my personal historic misunderstanding, or it was different 10 years ago.
• There is an interesting set of slides from Benjamin related to debuging input device issues.

From time to time I've to use chromium for creepy stuff like lifesize video conferencing with document sharing. The document sharing requires a chromium extension. Suddenly that stopped working today and I could not reinstall the extension. After trying a lot of stuff I had a look at the debian changelog and found out about:

chromium --enable-remote-extensions

See also #851927.

While reading the Tails 2.10 changelog I stumbled upon the fact that Tails now supports exFAT. Since Tails is Debian based I just checked the image and indeed it contains the fuse-exfat package. Do I've to assume that I've now another set of crosshairs on my back just because it's one possible maintainer you could attack to place malicious code into Tails? I'm not sure, and I'm also not sure if it would change much. I've always assumed to be a target just because I'm contributing to Debian, and because I'm working in IT operations. But to be honest so far my contributions to Debian are not on crucial packages and unexpected strange looking NMUs would always raise alarm bells for everyone. BTW the exfat fuse driver package builds reproducible. Maybe a good opportunity to thank the reproducible build team for this effort!

I started to reactivate my old moto g falcon during the last days of CyanogenMod in December of 2016. First step was a recovery update to TWRP 3.0.2-2 so I was able to flash CM13/14 builds. While CM14 nightly builds did not boot at all the CM13 builds did, but up to the last build wifi connections to the internet did not work. I could actually register with my wifi (Archer C7 running OpenWRT) but all apps claim the internet connection check failed and I'm offline. So bummer, without wifi a smartphone is not much fun. I was pretty sure that wifi worked when I last used that phone about 1.5 years ago with CM11/12, so I started to dive into the forums of xda-developers to look for alternatives. Here I found out about Exodus. I've a bit of trouble trusting stuff from xda-developer forums but what the hell, the phone is empty anyway so nothing to loose and I flashed the latest falcon build. To flash it I had to clean the whole phone, format all partitions via TWRP and then sideloaded the zip image file via adb (adb from the Debian/stretch adb package works like a charm, thank you guys!). Booted and b m wifi works again! Now Exodus is a really striped down mod, to do anything useful with it I had to activate the developer options and allow USB debugging. Afterwards I could install the f-droid and Opera apk via "adb install foo.apk". Lineage OS As I could derive from another thread on xda-developers Lineage OS has the falcon still on the shortlist for 14.x nightly builds. Maybe that will be an alternative again in the future. For now Exodus is a bit behind the curve (based on Android 6.0.1 from September 2016) but at least it's functional.

Just a short PSA for those around working with F5 devices: TMOS 11.6 introduced an experimental "mv" command in tmsh. In the last days we tried it for the first time on TMOS 12.1.1. It worked fine for a VirtualServer but a mv for a pool caused a sefault in tmm. We're currently working with the F5 support to sort it out, they think it's a known issue. Recommendation for now is to not use mv on pools. Just do it the old way, create a new pool, assign the new pool to the relevant VS and delete the old pool. Possible bug ID at F5 is ID562808. Since I can not find it in the TMOS 12.2 release notes I expect that this issue also applies to TMOS 12.2, but I did not verify that.

A friend of mine choose for $reasons to install the latest OpenSuSE 42.2 release as his new laptop operating system. It's been a while that I had contact with the SuSE Linux distribution. Must be around 12 years or so. The unsual part here is that I've to support a somewhat eccentric, but mostly ordinary user of computers. And to my surprise it's still hard to just plug in your existing stuff and expect it work. I've done so many dirty things to this installation in the last three days, my system egineering heart is bleeding. printing with a Canon Pixma iP100 printer This is a small portable Canon printer, about four years old. It provides a decent quality and its main strength is that it's small and really portable. Sadly the gutenprint driver just pushes through a blank page. No ink wasted on it at all. So the only reasonable other choice was a four year old binary rpm package provided by Canon. It has a file dependency on "libtiff.so.3" which is no longer available in recent GNU/Linux distributions. So I cheated and

- unpacked the tarball
- installed the rpm from the "packages" folder
zypper install cnijfilter-common-3.70-1.x86_64.rpm cnijfilter-ip100series-3.70-1.x86_64.rpm
... and choose to ignore the missing file dependency on libtiff.so.5.
ln -s /usr/lib64/libtiff.so /usr/lib64/libtiff.so.5
- re-ran the ./install.sh which registered the printer with cups and does whatever else
  magic is included in 1906 lines of shell.

To my surprise this driver still works and provides the expected quality. Though it's just a question of time until this setup will break. Be it an incompatible ABI change in libtiff or another lib in use by those Canon provided tools. QGIS and gdal with ECW support While the printer stuff is a rather common use case, having a map viewer for map files in the ECW format is the eccentric part. I found some hints on stackoverflow and subsequently https://trac.osgeo.org/gdal/wiki/ECW that a non-free library is required and a specific build of gdal. Then QGIS should be able to work with ECW files. Lucky us there is at least a OpenSuSE repository for gdal and QGIS. So I did the following:

zypper addrepo http://download.opensuse.org/repositories/Application:/Geo/openSUSE_Leap_42.2/Application:Geo.repo
zypper install qgis

Then I had to download the non-free ECW SDK from http://download.hexagongeospatial.com/downloads/erdas-ecw-jp2-sdk-v5.3-%28linux%29 - you'll and up with a '.bin' installer file. The installation process left me with "ERDAS-ECW_JPEG_2000_SDK-5.3.0" folder in my $HOME. I moved that one to /opt. Next step is adding the library to the ldconfig search path.

echo "/opt/ERDAS-ECW_JPEG_2000_SDK-5.3.0/Desktop_Read-Only/lib/x64/release/" > /etc/ld.so.conf.d/ecw.conf; ldconfig

Now it was "just" about rebuild gdal with ECW support. So I downloaded the required source packages with "zypper source-install gdal", edited the spec somewhere in "/usr/src/" to make the following modifications

--with-ecw=/opt/ERDAS-ECW_JPEG_2000_SDK-5.3.0/Desktop_Read-Only

added to the "./configure" invocation. And somewhere at the top we had to relax the requirement that all installed files have to be referenced inside the package.

%define _unpackaged_files_terminate_build 0

As a last step I had to "rpmbuild -ba" the package and force the installation via zypper once more, because this time we have a file depedency on the libecw stuff and it's obviously not listed in the rpm database. Last but not least I tried to put the gdal build on hold with

zypper addlock gdal libgdal20

to ensure it's not removed on the next update. Other non-free tools Beside of those two issues I had to install a range of other non-free tools, but currently they work without further issues or modifications. One is Teamviewer (i686 multiarch rpm) and the other one is XnViewMP. XnView is also able to show ECW files, but only the smaller ones. It crashes on bigger ones but that's also the case on Windows. Then there is also (required by some Italian map related websites) the ugly Adobe Flash Plugin for Firefox, but that one is sadly still a widespread issue. We also tried to try out the nvidia graphic drivers but at the moment we could only get the build in Intel card to work. Usually the preferred solution from my point of view but sometimes we see rendering glitches and I'm not sure if it's the driver or something else. my personal take away I hate to admit it but it's nothing extraordinary that was requested here. But still it took me the better part of two evenings to figure everything out. And even now it's not properly integrated and doomed to fail any day due to various updates and changes in the surounding ecosystem. I've full sympathy for every average user that would give up after two hours of research and try&error on this journey. For the printer drivers I'm happy to blame Canon. The printer situation as a whole improved from my point of view during the last decade, but it's still a pain in the ass with the very short shelf life you usually see with consumer models. For the ECW case one could discuss if it would be legally possible and helpful to do ugly dlopen() stuff to dynamcially load the shared libs. But then again someone has to make his hands dirty during the build and discussions about the legal use of header files will be the next chapter (hello Oracle). It's just ugly. Actually I know too little about the world of image formats to judge if someone has a good reason to keep this format commercial or not. From my personal point of view it's not useful and maybe even morally wrong. Technically one could argue if it would make sense to keep a local copy of the gdal build in "/opt" and start QGIS with a modified library path to prefer the private gdal build. Not sure if that is any better. On the other hand there are evolving mechanism like flatpack that would ease the handling of such situations. Buth then again we would be catering non-free software. It feels a lot like giving up. While my private working environment is except for firmware blobs free, I now created for someone a real "FrankenSuSE" to satisfy his everyday needs. On the one hand we now have another mostly satisfied user of a mostly free operating system. On the other hand that was only possible by adding a vast amount of non-free software. For sure we did not win the war, I'm not even sure if we've won a single battle here. It's just frustrating to see what is required to get someone up and running. With my personal attitude towards open source software it even feels wrong to invest so much time into fiddling with non-free components. What is still missing We currently lack an image viewer that allows us to print only a selection of an image, which is useful to print parts of a map. That usually works with XnView on Windows but does not work with the Linux version at the moment. I also tried gwenview and geeqie and had the same issue. Not sure if it's maybe a bug in XnView or one of the Qt parts (gwenview is also Qt based). I did not research that yet. Update: I spent quite some time looking into open bug reports for geeqie and gwenview. Seems the feature to print only a section of an image is something new. I've created #374299 (gwenview) and #457 (geeqie). For XnView I expect it's a difference between XnViewMP (the portable version) and the Windows only XnView Classic. Needs to be clarified and it might be worth to try XnView Classic with wine. Maybe printing with wine via cups works, I found at least some results for it on the internet.

No, not about the latest NOFX record, though it's a great one. Buy it. Took me a hell of a long time to get my head out of my arse and dive again into some Debian related work. Thanks to Nik for pushing me from time to time. So I've taken the time to upload LyX 2.2.2 to unstable and it's now build with Qt5. Afterall the package is still missing a lot of love, but I hope we've once again something for the upcoming stable release, that is close to the latest upstream stable release. If you use LyX please give it a try. For myself it's now the 6th year that I stopped using LyX after maintaining it for five years. And still I'm sponsoring the uploads and try to keep it at least functional. Strange how we sometimes take care of stuff even if we no longer have an active use for them.

Since I kind of recommended the free WoSign CA in the past, I would like to point out the issues that have piled up. Mozilla has a writeup due to a removal discussion for NSS: https://wiki.mozilla.org/CA:WoSign_Issues (Yes I'm late with this post, about a month or two by now ...) Since WoSign, or the person behind it, silently also bought StartCom we've now with StartSSL and WoSign two of the three free CAs in one hand with a questionable track record. That leaves everyone looking for a low budget option with Let's Encrypt.

What happened in the Reproducible Builds effort between Sunday September 11 and Saturday September 17 2016: Toolchain developments Ximin Luo started a new series of tools called (for now) debrepatch, to make it easier to automate checks that our old patches to Debian packages still apply to newer versions of those packages, and still make these reproducible. Ximin Luo updated one of our few remaining patches for dpkg in #787980 to make it cleaner and more minimal. The following tools were fixed to produce reproducible output:

• naturaldocs/1.51-2 by Petter Reinholdtsen, original patch by Chris Lamb.

Packages reviewed and fixed, and bugs filed The following updated packages have become reproducible - in our current test setup - after being fixed:

• elog/3.1.2-1-1 by Roger Kalt, original patch by Reiner Herrmann.
• eyed3/0.6.18-3 by Petter Reinholdtsen, original patch by Chris Lamb.
• frog/0.13.5-1 by Maarten van Gompel, original patch by Chris Lamb.
• gtranslator/2.91.7-3 by Andreas Henriksson, original patch by Reiner Herrmann.
• sozi/12.05-1.1 by Daniel Kahn Gillmor, original patch by Chris Lamb.

The following updated packages appear to be reproducible now, for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.)

• evince/3.21.92-1 by Michael Biebl.
• gnome-control-center/1:3.21.92-2 by Rapha l Hertzog.
• libipathverbs/1.3-2 by Ana Beatriz Guerrero Lopez.
• pagekite/0.5.8e-2 by Petter Reinholdtsen.

The following 3 packages were not changed, but have become reproducible due to changes in their build-dependencies: jaxrs-api python-lua zope-mysqlda. Some uploads have addressed some reproducibility issues, but not all of them:

• eurephia/1.1.0-6 by Alberto Gonzalez Iniesta, original patch by Chris Lamb.
• fdroidserver/0.7.0-1 by Hans-Christoph Steiner, original patch by Chris Lamb.
• mini-buildd/1.0.18 by Stephan S rken.
• nbc/1.2.1.r4+dfsg-3 by Petter Reinholdtsen, original patch by Chris Lamb.
• ncurses/6.0+20160910-1 by Sven Joachim, #818067 by Niels Thykier.
• python-kinterbasdb/3.3.0-4 by Santiago Vila, original patch by Chris Lamb.
• snapper/0.3.3-1 Hideki Yamane, original patch by Sascha Steinbiss.

Patches submitted that have not made their way to the archive yet:

• #838188 filed against ocaml by Johannes Schauer.

Reviews of unreproducible packages 462 package reviews have been added, 524 have been updated and 166 have been removed in this week, adding to our knowledge about identified issues. 25 issue types have been updated:

• Added a new annotation for issues called "fix-deterministic" to help us update package reviews more easily. This indicates whether we expect that an issue would always happen on Jenkins; i.e. if there is a successful build, then we know the issue is fixed for that package and can update our notes.
• Added random_order_in_sisu_javax_inject_named and too_much_input_for_diff.
• Removed timestamps_in_manpages_generated_by_ronn.
• Updated timestamps_in_allegro_dat_files. Additionally, 21 issues were marked with "fix-deterministic".

Weekly QA work FTBFS bugs have been reported by:

• Chris Lamb (10)
• Filip Pytloun (1)
• Santiago Vila (1)

diffoscope development A new version of diffoscope 60 was uploaded to unstable by Mattia Rizzolo. It included contributions from:

• Mattia Rizzolo:
  • Various packaging and testing improvements.
• HW42:
  • minor wording fixes
• Reiner Herrmann:
  • minor wording fixes

It also included from changes previous weeks; see either the changes or commits linked above, or previous blog posts 72 71 70. strip-nondeterminism development New versions of strip-nondeterminism 0.027-1 and 0.028-1 were uploaded to unstable by Chris Lamb. It included contributions from:

• Chris Lamb:
  • Testing improvements, including better handling of timezones.

disorderfs development A new version of disorderfs 0.5.1 was uploaded to unstable by Chris Lamb. It included contributions from:

• Andrew Ayer and Chris Lamb:
  • Support relative paths for ROOTDIR; it no longer needs to be an absolute path.
• Chris Lamb:
  • Print the behaviour (shuffle/reverse/sort) on startup to stdout.

It also included from changes previous weeks; see either the changes or commits linked above, or previous blog posts 70. Misc. This week's edition was written by Ximin Luo and reviewed by a bunch of Reproducible Builds folks on IRC.

... or the day I fixed my mail clients running on Debian/stretch. First of all mutt failed to open html mails or the html multipart stuff in Firefox. I found some interesting hints in a recent thread on debian-user. So now my "~/.mailcap" looks like this:

text/html; /usr/bin/firefox --new-tab %s;
text/html; /usr/bin/elinks -force-html -dump %s; copiousoutput

and I added the proposed "~/.muttrc" addition verbatim:

bind  attach  <return>  view-mailcap
alternative_order text/plain text/html
unauto_view *
auto_view text/html

For work related mails, where the use of html crap mails is a sad reality I can not avoid, I stick to Icedove. But beside of the many crashes everyone encountered recently it also crashes when I try to reach "Preferences -> Advanced -> Config Editor". So no chance to adjust the handling of http/https links in the UI. Luckily that configuration is still text, well XML, in a file called mimeTypes.rdf in in the profile directory. So I manually replaced "/usr/bin/iceweasel" with "/usr/bin/firefox" and a restart later clicking on http and https links works again. Yay.

Finally I got around taking a look at man libinput. And now with

Option "NaturalScrolling" "1"

in my xorg configuration multitouch scrolling works again in a natural way. What a relief, should've taken the 5 minutes to find that out a week ago.